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I THE CHANGING 
POLICY LANDSCAPE 

The American Recovery and Reinvestment Act 
(ARRA) of 2009 tunneled an unprecedented 
amount of federal funding to education 
initiatives through a variety of funding streams. 
By now, most education stakeholders are aware 
of the four primary assurances outlined in ARRA 
and made available to states through the Race 
to the Top competitive grant: 1 

• “Adopting standards and assessments 
that prepare students to succeed in 
college and the workplace to compete 
in the global economy.” 

• "Building data systems that measure 
student growth and success, and inform 
teachers and principals about how they 
can improve instruction.” 

• “Recruiting, developing, rewarding, and 
retaining effective teachers and principals, 
especially where they are needed most.” 

• “Turning around our lowest achieving 
schools” (U.S. Department of Education, 
2009, p. 2). 

Since the passage of ARRA, these assurances 
have driven changes in state legislation, 
especially as states prepared to participate in 
the Race to the Top competitive grant program. 

In a review of the 41 applications submitted for 
Phase I of Race to the Top, Learning Point 
Associates (2010b), an affiliate of American 
Institutes for Research, found that 29 (71 
percent) of the 41 applications submitted by 
states and the District of Columbia included 
descriptions of recently passed legislation or 
intentions to introduce legislation in support of 
Race to the Top program priorities. Specific to 
teacher evaluation, a total of 11 states passed, 
or expressed an intention to pass, legislation 
related to teacher evaluation in the following 
key areas: prescribing measures to evaluate 



teachers (7 states), prescribing the use of 
evaluation data (2 states), and prescribing both 
measures to evaluate teachers and the use of 
evaluation data (2 states). 

Although most new state laws focused on the 
use of student achievement data to assess 
teacher performance, another common theme 
in the legislation was the redesign of educator 
evaluation systems at the state and district 
levels, including the stated use of observation 
rubrics and other measures of teacher 
performance (Learning Point Associates, 2010b). 

In addition to ARRA, the Common Core State 
Standards movement, spearheaded by the 
National Governors Association (NGA) and the 
Council of Chief State School Officers (CCSSO), 
provides states with an additional incentive to 
agree on definitions for the essential 
knowledge and skills necessary to the future 
success of K-12 students. NGA and CCSSO 
worked collaboratively with states, educators, 
content experts, researchers, national 
organizations, and community groups to ensure 
that stakeholders had a significant role in the 
development process. Forty-one states, the 
District of Columbia, and the U.S. Virgin Islands 
have adopted the Common Core State 
Standards. 2 Currently, state standards are 
available in mathematics and English language 
arts, which also include literacy in history/ 
social studies, science, and technical subjects. 
NGA and CCSSO also consider the application 
of the standards to English learners and 
students with disabilities. 

This Policy-to-Practice Brief introduces five 
current examples of measures of teacher 
performance. The goal is to assist regional 
comprehensive centers and state education 
agencies in building local capacity to incorporate 
the use of alternative measures of teacher 
performance into the overhaul of state 
evaluation systems — especially in states with 
looming legislative deadlines. 



1 For a complete listing of education programs under ARRA as well as links to regulations, guidance, and resources 
provided by the U.S. Department of Education, visit http://www.ed.gov/recovery. 

2 For more information on the states and territories that have adopted the Common Core State Standards as well as links 
to the detailed standards, guidance, and other resources, visit http://www.corestandards.org. 



THE IMPORTANCE OF 
ALTERNATIVE 
MEASURES OF 
TEACHER 
PERFORMANCE 

ARRA and the Race to the Top grant program 
have pushed states and districts to invest 
in the development of high-quality teacher 
evaluation systems. Such systems have two 
specific elements: 

• A focus on student growth data as a measure 
of teacher effectiveness 

• Multiple measures to inform critical decisions 
relating to opportunities for teacher 
improvement and career advancement (e.g., 
promotion, tenure, equitable distribution, 
compensation). 

Historically, most states and districts have used 
classroom observations as the primary tool to 
assess teacher performance (Brandt, Thomas, 

& Burke, 2008; Weisberg, Sexton, Mulhern, 

& Keeling, 2009). Although classroom 
observations — in combination with student 
growth measures — provide multiple data points 
on teacher performance, additional alternative 
measures also should be considered to ensure 
a robust teacher evaluation system that 
captures the many facets of effective teaching. 

Alternative measures can take many forms, 
from student engagement surveys to teacher 
portfolios. It is beyond the scope of this brief 
to cover every alternative measure to assess 
teacher effectiveness; however, the brief 
highlights five measures that are included 
in the online Guide to Teacher Evaluation 
Products (National Comprehensive Center 
for Teacher Quality, 2010) as examples of 
alternative measures that have potential for 
use in teacher evaluation. 



In a review of teacher evaluation reforms 
proposed in state Phase 1 Race to the Top 
applications, Learning Point Associates (2010a) 
found that in addition to student growth 
measures, states also discussed plans to 
develop multiple measures of teacher 
performance beyond student learning. Although 
most state applications included references to 
observation rubrics, some states also described 
other measures of teacher performance in their 
applications, including the following (Learning 
Point Associates, 2010a): 

• A review of classroom artifacts or 
portfolios submitted by the teacher 

• Teacher planning, instructional, and 
assessment artifacts (6 states) 

• Teacher self-reflection portfolios (5 states) 

• Examples of student work (3 states) 

• Provisions for peer review and 
feedback (6 states) 

• Student reflections and feedback (5 states) 

• Teacher participation in professional 
development (1 state) 

• Follow-up work on teacher adaptation 
of classroom practices in response to 
feedback from formal and informal 
observations (1 state). 

Defining Effectiveness 

Understanding that student growth measures 
on their own have limitations for determining 
“effective” and "highly effective” designations for 
teachers and leaders, the U.S. Department of 
Education (2009) has reinforced the need 
to include multiple measures of teacher 
performance as the most robust approach 
to fully capturing classroom practice (See 
“Definitions of Effective and Highly 
Effective Teachers”). 
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The U.S. Department of Education (2009, p. 12) provides the following definitions of effective and highly 
effective teachers: 

Effective teacher means a teacher whose students achieve acceptable rates (e.g., at least one grade level 
in an academic year) of student growth (as defined in this notice). States, LEAs [local education 
agencies], or schools must include multiple measures, provided that teacher effectiveness is evaluated, in 
significant part, by student growth (as defined in this notice). Supplemental measures may include, for 
example, multiple observation-based assessments of teacher performance. 

Highly effective teacher means a teacher whose students achieve high rates (e.g., one and one-half 
grade levels in an academic year) of student growth (as defined in this notice). States, LEAs, or schools 
must include multiple measures, provided that teacher effectiveness is evaluated, in significant part, by 
student growth (as defined in this notice). Supplemental measures may include, for example, multiple 
observation-based assessments of teacher performance or evidence of leadership roles (which may 
include mentoring or leading professional learning communities) that increase the effectiveness of other 
teachers in the school or LEA. 



DEFINITIONS OF EFFECTIVE AND HIGHLY EFFECTIVE TEACHERS 



In its 2008 review of existing research on 
evaluating teacher effectiveness, the TQ Center 
introduced a five-point definition of teacher 
effectiveness that was intended to initiate 
state and regional conversations on the types 
of measures that might be needed to 
determine effective classroom teaching (Goe, 
Bell, & Little, 2008). The TQ Center’s definition 
recognizes the primacy of student growth data, 
but it also highlights additional important 
aspects of teaching, many of which are not 
currently measured through teacher 
observations or student learning growth 
measures. This definition highlights a specific 
need for alternative measures of teacher 
performance to determine effectiveness. 

Given the significant policy focus on reforming 
state and local teacher evaluation systems 
that include multiple measures of teacher 
performance, there is a clear need for 
the following: 

• The development of products and services 
that provide alternative measures of teacher 
performance 

• Widespread dissemination of the products 
and services for states to respond to 
legislative initiatives implemented since the 
passage of ARRA 



“The five-point definition of teacher 

effectiveness consists of the following: 

• Effective teachers have high expectations 
for all students and help students learn, 
as measured by value-added or other 
test-based growth measures, or by 
alternative measures. 

• Effective teachers contribute to positive 
academic, attitudinal, and social outcomes 
for students such as regular attendance, 
on-time promotion to the next grade, 
on-time graduation, self-efficacy, and 
cooperative behavior. 

• Effective teachers use diverse resources to 
plan and structure engaging learning 
opportunities; monitor student progress 
formatively, adapting instruction as needed; 
and evaluate learning using multiple 
sources of evidence. 

• Effective teachers contribute to the 
development of classrooms and schools 
that value diversity and civic-mindedness. 

• Effective teachers collaborate with other 
teachers, administrators, parents, and 
education professionals to ensure student 
success, particularly the success of 
students with special needs and those at 
high risk for failure.” (Goe et al., 2008, p. 8) 




I EXAMPLES OF ALTERNATIVE MEASURES 



States and districts recently have begun to implement teacher evaluation reforms. Table 1 provides 
information on five alternative measures of teacher performance that might be used to 
supplement growth measures and observation rubrics. (For additional information about these 
measures, refer to Appendixes A-E.) 

Table 1. Five Alternative Measures of Teacher Performance 



Alternative Measure 
Product or Service 
or Service 


Developer 


Type of Information Gathered 


Cost of Product 


Gallup Student Poll 


Gallup Inc. 

America’s Promise Alliance 

American Association of 
School Administrators 


The poll is administered to students 
in Grades 5-12. 

The poll measures three variables 
identified as key factors that drive 
students' grades: hope, engagement, 
and well-being. 


Registered public schools and 
districts can use this measure at 
no cost. 


Scoop Notebook 


National Center for 
Research on Evaluation, 
Standards, and Student 
Testing (CRESST) at the 
Center for the Study of 
Evaluation (CSE) 

RAND Corporation 
Stanford University 


This measure uses artifacts and 
related materials to represent 
classroom practice. 

Artifacts and other materials can 
include the following: lesson 
handouts; student classwork; 
homework; photos of classroom 
layout, equipment, and board work; 
teacher reflections on each lesson. 


States may use publically 
available research and resources 
to implement this measure in 
their schools at no cost. 

To receive expert assistance to 
use the tool, states may negotiate 
pricing with the developers. 


Surveys of Enacted 
Curriculum (SEC) 


Council of Chief State 
School Officers (CCSSO) 

Wisconsin Center for 
Education Research 
(WCER) 


Teachers report information on subject 
coverage, length of time spent on 
topics, and cognitive depth covered in 
their classroom instruction through an 
online survey. 

Teachers as well as school, district, 
and state leaders can use this 
information to inform professional 
development and assess the extent 
to which teacher instruction aligns 
with state standards and 
assessments. 


Cost for tools and services varies 
and is determined by CCSSO and 
WCER on a case-by-case basis. 


Teacher Portfolios 


Varies, based on specific 
example (See Appendix D.) 


Teachers pull together portfolios that 
can include the following: 

■ Video clips 

■ Lesson plans 

■ Teacher self-assessments or 
evaluations 

■ Examples of student work 


Costs vary, depending on whether 
portfolios are developed in-house 
or with consultant. (See Appendix 
D for more details.) 
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Alternative Measure 

Product or Service Developer Type of Information Gathered Cost of Product 

or Service 



Tripod Surveys 



Harvard University 



This measure consists of surveys for 
students, teachers, and parents. 

The surveys identify attitudes, 
perceptions, experiences, and 
classroom practice related to teacher 
content knowledge, pedagogical 
knowledge, and relationships between 
teachers and students. 

The surveys examine the Seven C’s 
of teacher quality: 

■ Care about students 

■ Control of student behavior 

■ Captivating students 

■ Clarifying lessons 

■ Challenging students 
academically 

■ Conferring with students 

■ Consolidating knowledge 



Costs vary, and consultation 
services are customized based 
on client needs. 



For additional information, please refer to the online Guide to Teacher Evaluation Products (www3.iearningpt.org/ 
tqsource/GEP/) and Appendixes A-E. 



The products and services included in Table 1 align with the types of alternative measures specified 
in some of the state Race to the Top applications, such as review of classroom artifacts or 
portfolios; teacher planning, instructional, and assessment artifacts; teacher self-reflection 
portfolios; examples of student work; provisions for peer review and feedback; and student 
reflections and feedback (Learning Point Associates, 2010a). The products and services were 
selected from the range of products available in more detail in the TQ Center's online Guide to 
Teacher Evaluation Products (2010). 




RESEARCH ON ALTERNATIVE MEASURES 



Although further evaluation and research is needed to fully understand the best way to fit these 
measures into teacher evaluation practices, Table 2 provides a short synopsis of the advantages and 
challenges identified by currently available research. 

Table 2. Advantages and Challenges of Alternative Measures 



Measure of Teacher 
Performance 


Research Cited* 


Advantages 


Challenges 


Gallup Student Poll 


America’s Promise Alliance 
(2010) 

Gallup Consulting Education 
Practice (2009) 

Lopez (2010) 

Lopez, Agrawal, and Calderon 
(2010) 


The poll is available through a 
secure, online administration 
website. 

Students can complete the poll in 
less than 10 minutes. 

For a fee, Gallup provides analysis of 
the data that correlate survey results 
with grade-level or classroom-level 
gains. 


The poll is not an alternative 
measure for all students, as it is 
not available before Grade 5. 

The poll requires Internet access. 


Scoop Notebook 


Borko, Stecher, Alonzo, 
Moncure, and McClam (2005) 

Borko, Stecher, and Kuffner 
(2007) 

Stecher etal. (2005) 


This measure can increase teacher 
commitment to the evaluation 
process. 

Schools and districts may be able 
to better address the professional 
development needs of teachers with 
the critical information gleaned from 
this measure. 

This measure may assist teachers 
in analyzing student work in 
professional learning communities. 


Only mathematical and science 
rating guides are currently 
available. 

It might be difficult to develop 
as a rigorous and comparable 
measure of teacher effectiveness 
as part of a high-stakes 
evaluation system. 

It may not be useful as a 
measure in classrooms that 
produce minimal artifacts (e.g., 
physical education). 

This approach takes time and 
effort to complete. 


Surveys of Enacted 
Curriculum 


Blank (2004) 

Blank, Porter, and Smithson 
(2001) 

Council of Chief State School 
Officers (2004) 

Council of Chief State School 
Officers (2010) 


The SEC collect a large amount of 
information on teacher practice. 

The SEC report on instructional 
practice across a school year, which 
can be difficult information to obtain 
through other types of evaluation 
measures. 


This measure relies on teacher 
self-reporting, which may not be 
accurate. 

This measure requires training 
for teachers and administrators 
to view and understand the data 
to be used most effectively. 
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Measure of Teacher 
Performance 


Research Cited* 


Advantages 


Challenges 


Teacher Portfolios 


Goe, Bell, and Little (2008) 
Little, Goe, and Bell (2009) 

National Comprehensive 
Center forTeacher Quality 
(2010) 


Teachers collect and reflect on 
evidence across various activities, 
which encourages a perspective on 
teaching beyond the classroom. 

If conducted collaboratively, this 
measure can create a more cohesive 
teaching team. 

Receiving and providing support 
to colleagues may promote 
professional growth. 

This measure can be conducted 
in an online format or through a 
physical collection of artifacts. 


Feedback is time-sensitive. 

It is best to apply this measure 
over the course of a year; 
however, it is difficult to regulate. 

There is tension between using 
evidence as part of an 
evaluation or for professional 
growth. 


Tripod Surveys 


Bill & Melinda Gates 
Foundation (2010) 

Ferguson (2002a) 
Ferguson (2002b) 


This measure can be used to report 
otherwise unobservable factors that 
may affect teaching, such as 
knowledge, intentions, expectations, 
and beliefs. 

The surveys provide the unique 
perspective of the teacher as well 
as the perspective of students, 
who have the greatest amount of 
experience with teachers. 

This measure can provide formative 
information to help teachers improve 
practice in a way that connects with 
students. 


This measure relies on teacher 
self-reporting, which may not be 
accurate. 

Students cannot provide 
information on certain aspects 
of teaching, such as a teacher's 
content knowledge, curriculum 
fulfillment, or professional 
activities. 






This measure makes use of the 
perspective of students who may 
be as capable as adult raters of 
providing accurate ratings. 





*For full references, see Appendixes A-E. 



As evidenced in Table 2, each measure has distinct advantages and implementation challenges. 
In some cases, such as the Gallup Student Poll and the Tripod Surveys, the relatively small cost of 
implementation is advantageous. However, it is also important to take into account the state’s or 
district’s specific teacher evaluation needs. 
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■ CONCLUSION 

As state and district efforts continue to focus 
on teacher evaluation system reform, it is 
necessary to explore options for the gradual 
inclusion of multiple measures of performance 
to accurately evaluate teacher effectiveness. As 
state and district staff consider the five 
alternative measures presented in this brief, 
they should reflect on the following questions: 

• What teaching standards is the system 
trying to measure? 

• What kind of support can the state provide 
to LEAs for implementation? 

• How will the evaluation system be used? 

■ Guiding professional development 

■ Certification or tenure decisions 

■ Teacher career ladders 

■ Alternative compensation programs 

■ Addressing the inequitable distribution 
of teachers 



For a more in-depth look at making decisions 
regarding state and district teacher evaluation 
systems, see the Practical Guide to Designing 
Comprehensive Teacher Evaluation Systems 
(http://www.tqsource.org/publications/ 
practicalGuideEvalSystems). This guide 
walks states and districts through questions 
that are essential to the development 
and implementation of a high-quality, 
comprehensive teacher evaluation system. 

The advantages and implementation challenges 
of the alternative measures presented in this 
brief directly relate to the type of outcomes 
affected by the evaluation system. States and 
districts should carefully review examples of 
each measure in practice and determine the 
appropriate measures in the context of their 
school systems. 
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APPENDIXES 



APPENDIX A. GALLUP STUDENT POLL 



Developer of Product and Services 

The Gallup Student Poll was designed by Gallup Inc., in partnership with America’s Promise Alliance 
and the American Association of School Administrators. 



Description of Product and Services Available 

In 2009, Gallup Inc. launched the Gallup Student Poll, a school-based online survey for students in 
Grades 5-12 that measures three variables: hope, engagement, and well-being. Gallup Inc. defines 
hope as “the ideas and energy students have for the future,” engagement as a student’s “level of 
involvement in and enthusiasm for school,” and well-being as “how students think about and 
experience their lives” (see America’s Promise Alliance, 2010, listed in the Research and Resources 
section at the end of this appendix). Through extensive research, these three variables were identified 
as key factors that drive students' grades, achievement scores, retention, and future employment. 
Furthermore, research has revealed that the variables are linked to teacher talent and teacher 
engagement; staff and student engagement have been shown to drive positive outcomes and explain 
variance in school performance (see Gallup Consulting Education Practice, 2009, listed in the 
Research and Resources section). 

The survey is administered once during each school year. Students can access the survey on a secure 
website using a registered account. The survey takes, on average, less than 10 minutes to complete. 
In addition to several demographic questions (e.g., age, grade, gender), students are asked 20 core 
questions about their perspectives related to their home, school, and community lives. Survey 
questions were first developed in 2006 and have since been reviewed and refined based on 
additional research, focus group feedback, and psychometric studies conducted from 2008 to 2010. 
Studies include a 2008 expert review of items, pilot studies in 2008 and 2009, representative panel 
studies in 2009 and 2010, and a 2009 validation study. 

In 2009 and 2010, more than 450,000 students from across the country took the survey. Data from 
the survey have been used by schools and districts to build student and staff engagement and to 
provide information on how to select strategic initiatives, trainings, and interventions. 



Training for Use of Product and Services 

Gallup Inc. has developed a webinar series to communicate information about the Gallup Student 
Poll to educators and community leaders. The webinars are free and are offered throughout the year. 
For a schedule of upcoming webinars, please visit the Online Learning & Webinars webpage 

(www.gallupstudentpoll.com/121688/Online-Learning-Webinars.aspx). 



Cost of Product and Services 



The survey is free for registered public schools and districts. 



Advantages and Implementation Challenges 



Advantages 

• Free of charge. 

• Available online through a secure website. 

• Takes less than 10 minutes to complete. 



Implementation Challenges 

• Not available for students prior to Grade 5. 

• Requires computers with Internet access. 



How States Can Get More Information 



States can get more information at the Gallup Student Poll website (www.gallupstudentpoll.com). 
Technical support, provided by Gallup Inc. is available by phone (866-346-4408) Monday through 
Thursday from 7 a.m. to 7 p.m. and Friday from 7 a.m. to 5 p.m. (Central Time). 

Research and Resources 

America's Promise Alliance. (2010, August 12). Gallup student poll finds gap between perception and 
reality in youth hope, engagement and wellbeing [Press release]. Retrieved May 6, 2011, from 

http://www.americaspromise.org/About-the-Alliance/Press-Room/Press-Releases/2010/Gallup- 

Student-Poll-Results.aspx 

Gallup Consulting Education Practice. (2009). Building engaged schools: A scientific method for 
improving school performance [Brochure]. Washington, DC: Author. Retrieved May 6, 2011, from 

http://www.gallup.com/consulting/education/File/116839/Engaged_Schools_Brochure.pdf 

Lopez, S. J. (2010). Youth readiness for the future: A report on findings from a representative Gallup 
Student Poll sample. Washington, DC: Gallup Inc. Retrieved May 6, 2011, from http://www.gallup. 
com/poll/File/141842/Youth%20Readiness%20For%20the%20Future,%20August%202010.pdf 

Lopez, S. J., Agrawal, S., & Calderon, V. J. (2010). The Gallup Student Poll technical report. 

Washington, DC: Gallup Inc. Retrieved May 6, 2011, from http://www.gallupstudentpoll.com/ 
File/141995/Student-Poll_Technical_Report_August_2010.pdf 



APPENDIX B. SCOOP NOTEBOOK: EXAMINING 
CLASSROOM ARTIFACTS 



Developer of Product and Services 

The Scoop Notebook was developed by the National Center for Research on Evaluation, Standards, 
and Student Testing (CRESST) at the Center for the Study of Evaluation (CSE); RAND Corporation; and 
Stanford University. 



Description of the Product and Services Available 

The Scoop Notebook is a protocol for gathering and rating the quality of middle school mathematics 
and science classroom artifacts. It was developed through a five-year project funded through CRESST. 
The goal of the project was to use artifacts and related materials to represent classroom practice well 
enough that a person unfamiliar with a teacher or lessons can make valid judgments about selected 
features of practice solely on the basis of those materials. Moreover, there are two potential uses of 
the Scoop Notebook: as part of a system of multiple measures to characterize teacher effectiveness 
or as a formative tool for teacher professional development. 

During the course of one week, teachers collect artifacts and other materials (e.g., lesson handouts; 
student classwork; homework; photos of classroom layout, equipment, and board work; teacher 
reflections on each lesson) and put them in a binder called the “Scoop Notebook.” (Articles and 
studies listed in the Research and Resources section at the end of this appendix provide detailed 
instructions on creating the binders and using rubrics to analyze artifacts.) Rating guides for the 
notebook are based on previous research, the National Science Education Standards, and Principles 
and Standards for School Mathematics. Although the tool was developed and field-tested in middle 
school classrooms, the developers believe it is appropriate for other grade levels as well. 

During the five-year project, developers have conducted numerous studies to develop, refine, and test 
the reliability and validity of the product (see the Research and Resources section at the end of this 
appendix). Between 2003 and 2007, the Scoop Notebook was tested and used successfully in 36 
middle schools in Los Angeles and Denver. Because the tool is publicly available, it may be used 
in multiple schools and districts beyond the developers’ knowledge. 



Training for Use of Product and Services 

The Scoop Notebook can be used without training. All materials and guidebooks are available online 
at no cost to the user. Questions concerning the specific use of the notebook can be addressed to 
the developers listed in the How States Can Get More Information section. 

Cost of Product and Services 

States may use publically available research and resources (see the Research and Resources 
section) to implement the Scoop Notebook in their schools, free of cost. To receive expert 
assistance to use the tool, states may negotiate pricing with the developers (see the How States 
Can Get More Information section). 



Advantages and Implementation Challenges 



Advantages 



Implementation Challenges 



• Free online; additional expert assistance 
available for a fee. 



• Currently, only mathematics and science 
ratings guides available. 



• May increase teacher commitment to 



• May be difficult to develop as a rigorous and 
comparable measure of teacher 
effectiveness. 



the evaluation process. 



• May provide schools and districts critical 
information to better address professional 
development needs of teachers. 



• May not be useful for a measurement of 
classrooms that produce minimal artifacts 
(e.g., physical education). 



• May assist teachers in analyzing student 
work in professional learning committees 



• Takes time and effort to complete. 



How States Can Get More Information 

States can get more information from the developers of the product: 

• Dr. Hilda Borko (650-723-7640, hildab@stanford.edu) 

• Dr. Brian Stecher (310-393-0411, brian_stecher@rand.org) 

Research and Resources 

Borko, H., Stecher, B. M., Alonzo, A. C., Moncure, S., & McClam, S. (2005). Artifact packages for 
characterizing classroom practice: A pilot study. Educational Assessment, 10(2), 73-104. 

Borko, H., Stecher, B., & Kuffner, K. (2007). Using artifacts to characterize reform-oriented instruction: 
The Scoop Notebook and rating guide (CSE Technical Report No. 707). Los Angeles: National 
Center for Research on Evaluation, Standards, and Student Testing (ERIC Document 
Reproduction Service No. ED495853). Retrieved May 6, 2011, from http://www.eric.ed.gov/ 
PDFSZED495853.pdf 

Stecher, B., Wood, A. C., Gilbert, M. L., Borko, H., Kuffner, K. L., Arnold, S. C., et al. (2005). Using 
classroom artifacts to measure instructional practices in middle school mathematics: A two-state 
field test (CSE Report No. 662). Los Angeles: National Center for Research on Evaluation, 
Standards, and Student Testing. Retrieved May 6, 2011, from http://www.cse.ucla.edu/products/ 
reports/r662.pdf 



APPENDIX C. SURVEYS OF ENACTED CURRICULUM 



Developer of Product and Services 

The Surveys of Enacted Curriculum (SEC) were developed by the Council of Chief State School Officers 
(CCSSO) and the Wisconsin Center for Education Research (WCER). 



Description of Product and Services Available 

The SEC are online surveys that ask teachers to report information on subject coverage, length of 
time spent on topics, and cognitive depth covered in their classroom instruction. Teacher results can 
be compared with the content included in state standards and state assessments. 

Using aggregated information from several teachers, administrators at the school, district, and state 
levels can identify the extent to which teacher instruction aligns with state standards and state 
assessments and use this information to inform professional development and school improvement. 
By tracking this information over time, the SEC can provide feedback to schools, districts, and states 
on program implementation. 

Individual teachers also can review their practice and compare it with standards and the results of 
other teachers in their school or district. Consequently, it is possible for SEC data to be part of the 
information that teachers consider when self-evaluating their performance. 

This tool was designed for Grades K-12 mathematics, science, and language arts teachers. 
Mathematics and science surveys were written and field-tested from 1994 to 1998, with English 
language arts surveys and reports developed from 2002 to 2003. Eleven states are part of the SEC 
State Collaborative on Assessment and Student Standards: Arkansas, Idaho, Indiana, Iowa, Kansas, 
Mississippi, Missouri, Montana, Ohio, Oregon, and Wisconsin. 

The final report of the SEC, a study of the mathematics and science measures across 11 states, was 
funded through a grant by the National Science Foundation and published in 2001. It includes 
information on measure validity and ways to mitigate issues related to teacher self-reporting on 
practice (see Blank, Porter, & Smithson, 2001, listed in the Research and Resources section at the 
end of this appendix). 

The Common Core State Standards recently were analyzed for their content, and the results are 
publically available (see Council of Chief State School Officers, 2010, listed in the Research and 
Resources section). Several states are currently using the SEC to consider the alignment between 
instruction and the Common Core State Standards. Currently, SEC instruments are being adapted and 
expanded to facilitate a deeper examination of the instruction that students with disabilities receive. In 
addition, there are plans to develop a teacher-log format as well as a tool that would allow teachers 
to study the intended curriculum as compared with the enacted and assessed curriculum. 



Training for Use of Product and Services 



Training can be scheduled by contacting CCSSO or WCER. Resources related to training can be found 

at the SEC Resources webpage (seconline.wceruw.org/resources.asp). 



Cost of Product and Services 

Costs of tools and services vary and can be determined by contacting CCSSO or WCER. 



Advantages and Implementation 

Advantages 

• Collects a large amount of information on 
teacher practice. 

• Reports on instructional practice across a 
school year — information that is difficult to 
obtain through other types of evaluation 
measures. 

How States Can Get More Information 

States can find more information at the CCSSO SEC webpage (www.secsurvey.org) and the WCER 

SEC webpage (seconiine.wceruw.org/secWebHome.htm) or by contacting the following: 

• Rolf K. Blank (202-336-7044; rolfb@ccsso.org) 

• John Smithson (608-263-4354; johns@education.wisc.edu) 



Challenges 

Implementation Challenges 

• Relies on teacher self-reporting, which may 
not be accurate. 

• Requires training for teachers and 
administrators to view and understand the 
data so they may be used most effectively. 



Research and Resources 

Blank, R. K. (2004, April). Findings on alignment of instruction using enacted curriculum data: Results 
from urban schools. Paper presented at the annual meeting of the American Educational 
Research Association, San Diego, CA. Retrieved May 6, 2011, from http://seconline.wceruw.org/ 
Reference/BlankAlignmentPaper AERA04.doc 

Blank, R. K., Porter, A., & Smithson, J. (2001). New tools for analyzing teaching, curriculum and 
standards in mathematics and science: Results from Survey of Enacted Curriculum Project final 
report. Washington, DC: Council of Chief State School Officers. Retrieved May 6, 2011, from 

http://seconline.wceruw.org/Reference/SECnewToolsreport.pdf 

Council of Chief State School Officers. (2004). Data on enacted curriculum study: Summary of 
findings. Washington, DC: Author. 

Council of Chief State School Officers. (2010, September 20). Content analysis of Common Core 
State Standards: Initial findings [PowerPoint presentation]. Retrieved May 6, 2011, from 

http://www.ccsso.org/Documents/2010/News/CCSS%20anaiysis%20webinar%209%2020rev%20 

final%>20.ppt 



APPENDIX D. TEACHER PORTFOLIOS 



Developer of the Product and Services 

Teacher portfolios have been developed by various state education agencies, local education 
agencies, and education organizations. 



Description of the Product and Services Available 

Following are some examples of teaching portfolios. 

Washington ProTeach Portfolio 

The ProTeach portfolio collects the following student-based evidence to measure teacher effectiveness: 

• Professional growth and contributions. Includes analysis and reflection on professional growth 
and its impact on student learning. 

• Building a learning community. Includes a description and analysis of the learning environment 
established in the single class or classroom. 

• Curriculum, instruction, and assessment. Includes analysis and reflection of the curriculum, 
instruction, and assessment and their impact on three focus students. 

Artifacts collected for the portfolio include teacher and student work, written commentary, and 
samples in student voice (e.g., evidence of student learning from the students' perspective). 

Alexandria (Virginia) City Public Schools — Performance Evaluation Program 

The Performance Evaluation Program has four components: formal observations, informal 
observations, teacher portfolios, and academic goal-setting. The teacher portfolios are made up 
of artifacts that provide documents for 17 performance responsibilities, determined by Alexandria 
City Public Schools. 

Performance Assessment for California Teachers — Teaching Event 

Teaching Event is a teacher portfolio modeled after the teacher portfolio assessments of the 
Connecticut Department of Education, Interstate Teacher Assessment and Support Consortium, and 
National Board for Professional Teaching Standards. It documents work that meets criteria for six 
components: context, planning, instruction, assessment, reflection, and academic language. The goal 
is to have teacher candidates make connections between the different tasks and to provide evidence 
from a brief learning segment in depth. The directions for constructing the Teaching Event portfolio are 
designed to direct teacher candidates to plan, teach, and reflect on their teaching within the specific 
context of their students and their learning. Teaching Event portfolios include video clips, scorers with 
subject-specific expertise, and subject-specific benchmarks. Training is provided on its use. 



National Board for Professional Teaching Standards — National Board Certification 

National Board Certification is a standards-based assessment of teacher effectiveness. A score 
reflects the degree to which assessors were able to locate clear, consistent, and convincing evidence 
that the candidate has met the standards specific to his or her certificate field. The National Board 
Certification process consists of a teacher portfolio as well as other components. The portfolios are 
required to contain four entries. Three of these entries are classroom based; the fourth requires 
working with families and the larger community and with colleagues and the larger profession. At 
least two of the classroom-based entries must use video recording. In addition, teachers must 
provide a collection of student work as well as commentary describing, analyzing, and reflecting on 
the evidence. 

The National Board for Professional Teaching Standards has conducted analyses every year to 
determine the level of assessor reliability. These analyses indicate that assessors are making reliable, 
accurate, and fair evaluations of candidates’ responses. The standards committees recommend to the 
National Board the specific standards for each certificate area and advise those involved in developing 
the corresponding assessment. The standards and the certificates are structured along two 
dimensions: the developmental level of students and the subject area. 

Kansas Performance Teaching Portfolio 

The Kansas Performance Teaching Portfolio (KPTP) requires teachers to provide information about the 
unit's lesson plans and assessments. Specific information about how the instruction is modified for 
two individual students within the classroom also is required. In addition, the teacher candidate 
reflects on the implementation of the unit for the whole class and the two focus students. The 
portfolios must address six focus areas: 

• Analysis of contextual information 

• Analysis of learning environment factors 

• Instructional implementation 

• Analysis of classroom learning environment 

• Analysis of assessment procedures 

• Reflection and self-evaluation 

KPTP measures the teacher candidate’s ability to design, deliver, and reflect on an entire unit of study 
through four distinct sources of evidence: 

• Contextual information and learning environment factors 

• Designing instruction 

• Teaching and learning 

• Reflection and professionalism 



Training for Use of Product and Services 

The available training for use of these products and services varies, depending on whether the 
state developed the rubrics in-house or used outside consulting services. The TQ Center's 
Guide to Teacher Evaluation Products (www3.learningpt.org/tqsource/GEP/) provides additional 
information for each example. 



Cost of Product and Services 



The costs of these products and services vary, depending on whether the state developed the rubrics 
in-house or used outside consulting services. The TQ Center’s Guide to Teacher Evaluation Products 
(www3.learningpt.org/tqsource/GEP/) provides additional information for each example. 



Advantages and Implementation Challenges 



Advantages 

• Evidence across various activities collected 
and considered by teachers, which 
encourages a perspective on teaching beyond 
the classroom. 

• Potential for a more cohesive teaching team 
if the approach is applied collaboratively. 

• May promote professional growth through 
provision of support to colleagues. 



Implementation Challenges 

• Time-sensitive feedback. 

• Best when applied over the course of a year 
but difficult to regulate. 

• Tension between using evidence as part of an 
evaluation and for professional growth. 



How States Can Get More Information 

• Washington ProTeach Portfolio: www.waproteach.org 

• Alexandria (Virginia) City Public Schools Performance Evaluation Program (PEP): www.ascd.org/ 
publications/books/104136/chapters/Assessing_Teacher_Quality_Through_Goal-Setting@_The_ 
Alexandria, _Virginia,_Schooi_District.aspx 

• PACT Assessment — Teaching Event: www.pacttpa.org/_main/hub. php?pageName=Home 

• National Board for Professional Teaching Standards: www.nbpts.org 

• Kansas Performance Teaching Portfolio: www.ksde.org/Default.aspx?tabid=3769 



Research and Resources 

Goe, L., Bell, C., & Little, 0. (2008). Approaches to evaluating teacher effectiveness: A research 

synthesis. Washington, DC: National Comprehensive Center for Teacher Quality. Retrieved May 6, 
2011, from http://www.tqsource.org/publications/EvaluatingTeachEffectiveness.pdf 

Little, 0., Goe, L., & Bell, C. (2009). A practical guide to evaluating teacher effectiveness. Washington, 
DC: National Comprehensive Center for Teacher Quality. Retrieved May 6, 2011, from 

http://www.tqsource.org/publications/practicalGuide.pdf 

National Comprehensive Center for Teacher Quality. (2010). Guide to teacher evaluation products 
[Website]. Washington, DC: Author. Retrieved May 6, 2011, from http://www3.learningpt.org/ 
tqsource/GEP/ 



APPENDIX E. TRIPOD SURVEYS: 

STUDENT, TEACHER, AND PARENT SURVEYS 

Developer of Product and Services 

The Tripod Surveys were developed by Ron Ferguson, Ph.D., at Harvard University, and 
Cambridge Education. 



Description of Product and Services Available 

Tripod surveys are one component of the Tripod Project, which aims to improve school capacity to 
address content, pedagogy, and relationships (the “tripod” of quality teaching) while closing 
achievement gaps. The surveys are available for students, teachers, and parents. Tripod surveys 
identify attitudes, perceptions, experiences, and practices in classrooms as they relate to the content 
knowledge of teachers, the pedagogical knowledge of teachers, and the relationships between 
teachers and students. 

Tripod surveys examine the Seven C’s of quality teaching: care about students, control of student 
behavior, captivating students, clarifying lessons, challenging students academically, conferring with 
students, and consolidating knowledge. Tripod surveys are now in their 11th version. Previous 
research indicates that classrooms with high student ratings on the Seven C’s also produced higher 
average gains in student achievement. Currently, a modified version of the Tripod student survey is 
being used as part of the Measures of Effective Teaching (MET) Project funded by the Bill & Melinda 
Gates Foundation, which is researching the classroom practice of more than 3,000 teachers. 

The Tripod student, teacher, and parent surveys were developed for use with teachers in any subject 
or grade level. The Tripod Project is now offering value-added analysis, using results from Tripod 
surveys to predict student achievement on state tests. 



Training for Use of Product and Services 

Resources and research on the Tripod Project can be found at the Materials Archive webpage 

(www.tripodproject.org/index.php/materials/materials_overview/). 

Cost of Product and Services 

The Tripod Project offers consulting and support for student, teacher, and parent surveys; analysis 
and reporting; strategic school improvement planning; and professional development. Consultation 
services are customized based on client needs. For more information, see the Services and Offerings 
webpage (www.tripodproject.org/index.php/services/services_overview/). 



Advantages and Implementation Challenges 



Advantages 

• Can be used to report otherwise unobservable 
factors that may affect teaching, such as 
knowledge, intentions, expectations, and beliefs. 

• Provides the unique perspective of the teacher. 

• Provides the perspective of students, who have 
the greatest amount of experience with 
teachers. 

• Can provide formative information to help 
teachers improve practice in a way that will 
connect with students. 

• Makes use of the perspectives of students, who 
may be as capable as adult raters at providing 
accurate ratings. 



Implementation Challenges 

• Relies on teacher self-reporting, which may 
not be accurate. 

• Should not be used as the sole or primary 
measure of teacher evaluation because 
student ratings have not been validated for 
use in summative assessment. 

• Information on aspects of teaching (e.g., a 
teacher’s content knowledge, curriculum 
fulfillment, or professional activities) not 
available from students. 



How States Can Get More Information 

States can find more information at the Tripod Project website (www.tripodproject.org) or by contacting 
Rob Ramsdell (rob.ramsdell@camb-ed-us.com). 



Research and Resources 

Bill & Melinda Gates Foundation. (2010). Student perceptions and the MET Project. Seattle, WA: 
Author. Retrieved May 6, 2011, from http://metproject.org/downloads/Student_ 
Perceptions_092110.pdf 

Ferguson, R. F. (2002a). Addressing racial disparities in high-achieving suburban schools. NCREL 
Policy Issues, 13. Retrieved May 6, 2011, from http://www.ncrel.org/policy/pubs/pdfs/pivoll3.pdf 

Ferguson, R. F. (2002b). What doesn’t meet the eye: Understanding and addressing racial disparities 
in high-achieving suburban schools. Cambridge, MA: Harvard University, Wiener Center for Social 
Policy. Retrieved May 6, 2011, from http://www.tripodproject.org/uploads/file/What_doesnt_ 
meet_the_eye.pdf 
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